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Foreword 



The Division of Science Resources Studies (SRS) of the National Science Foundation (NSF) 
supported the workshop on “Strategic Research Partnerships,” that was convened at SRI, 
International, Washington D.C., on October 13, 2000. The objective of this workshop was to identify 
for NSF, SRS, and the National Science Board the policy needs for, and available data on, indicators 
related to the formation, activities, and economic consequences of alliances and strategic research 
partnerships (SRP). An SRP was broadly defined as an innovation-based relationship that involves, 
at least partly, a significant effort in research and development. 

Patterns of the Nation’s R&D investments have changed considerably during the past twenty years, 
including an apparent increase in the web of partnerships among firms, universities, and federal 
agencies in conducting R&D and sharing the resultant knowledge. The information generated by this 
workshop is intended to assist NSF in its continuing effort to track and measure these structural and 
process changes, and to advance the possibility of developing indicators related to SRPs. An 
understanding of the nature and magnitude of these changes is needed to inform program and policy 
deliberations and assist in planning various science and technology initiatives. 

The information presented here complements and augments the coverage on R&D partnerships and 
technology alliances in the Science and Engineering Indicators report (Chapter on U.S. and 
International Research and Development Funds and Alliances) of the National Science Board. 



Lynda T. Carlson 
Division Director 

Division of Science Resources Studies 

Directorate for Social, Behavioral, and Economic Sciences 

John E. Jankowski 
Program Director 

Research and Development Statistics Program 
July 2001 
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Strategic Research Partnerships: Results of the Workshop 



Albert N. Link 

University of North Carolina at Greensboro 

Nicholas S. Vonortas 
The George Washington University 



I. Introduction 

A strategic research partnership (SRP) can be broadly defined as an innovation-based relationship that 
involves, at least partly, a significant effort in research and development. The objectives of this 
workshop on strategic research partnerships were to evaluate: 

• What are the policy needs for indicators related to the formation, activities, and economic 
consequences of alliances and SRPs? What data and indicators are currently available about 
alliances and SRPs? What are their strengths and weaknesses? 

• How should the Science Resources Studies (SRS) Division of NSF proceed to develop SRP 
indicators? 

This paper summarizes the salient points of the background papers that follow in this report, as well 
as the general discussion during the workshop. To advance NSF’s interest in the possibility of 
developing indicators related to SRPs, the papers and the discussion appraised the potential use and 
the nature of such indicators. The approach was essentially based on two questions: 

• Are SRP’s important to the economic system? More specifically, do SRPs deserve policy 
attention? Are SRPs important from an economic or policy perspective? 

• What data initiatives are needed with regard to understanding the economic importance or policy 
relevance of SRPs? 

There is little doubt as to the answer to the first question. Prior research has shown that SRPs 
constitute an important-and probably increasing-component of the innovation system, and the papers 
presented and discussed at the workshop confirm this. There is a long list of reasons why this is so. 
In a few words, it can be argued that SRPs are socially useful because they expand the effective R&D 
resources applied to innovative investment. 

Second, the background papers and the discussion during the workshop clearly indicated that existing 
data on SRPs suffer from various shortcomings. There is a need to develop systematic tracking of the 
incidence of the inputs and outputs associated with various types of SRPs! 

Analytical and policy needs have traditionally driven the construction of new statistical indicators. 
Specific broad policy questions apply pressure and “pull” such indicators. The literature and, more 
generally, the understanding of a phenomenon is mature enough to be able to support 
methodologically the construction of efficient indicators. It was the consensus of opinion at the 
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workshop that this is exactly where we now are with respect to SRPs; SRS should begin a systematic 
collection of statistical information related to various dimensions of SRP activity. 

Section II of this paper summarizes the chronology of innovation-related indicators that NSF has 
developed. Section III presents our interpretative summary of the salient points from the background 
papers presented at the workshop and the discussion that followed as related to the first of the two 
objectives of this workshop: What are the policy needs for indicators related to the formation, 
activities, and economic consequences of alliances and SRPs? What data and indicators are 
currently available about alliances and SRPs? What are their strengths and weaknesses? Section 
IV summarizes the workshop discussion that was related to the second of the two objectives: How 
should the Science Resources Studies (SRS) Division of NSF proceed to develop SRP indicators? 

II. Chronology of Related Indicators by NSF 

NSF has had significant experience with indicators related to strategic research partnerships, R&D 
collaboration, and technology transfer. The chronology of the introduction of these indicators is: 

• R&D expenditure flows across sectors (1972) 

• Bibliometric co-authorship indicators (1980) 

• University patents ( 1 982) 

• Federal technology transfer indicators (1991) 

(CRADAs, patents, licenses and invention disclosures) 

• Industrial joint research ventures (1991) 

• R&D expenditure flows across countries (1991) 

• International strategic alliances (1993) 

• University-industry research centers (1993) 

• Foreign location of R&D facilities (1993) 

• Federal cooperative technology program dollar support (1996) 

• University technology transfer indicators (1996) 

(AUTM licenses, startups, revenue) 

• Patent/article citation indicators (1996) 

• Defense dual use programs ( 1 996, 1 998) 

• Advanced Technology Program data ( 1 998) 

III. Overview of Background Papers 

Nine background papers were commissioned and presented at the workshop. The salient points from 
these papers are: 

• The mainstream theoretical economic literature may not be very useful in isolation in indicating 
the best approach to constructing SRP indicators. While industrial organization economists were 
pioneers in the early 1980s in getting a better understanding of the spreading phenomenon of 
inter-organizational cooperation, they have paid less attention to formally modeling the important 
understandings about the variety of forms of SRPs and the variety of incentives to join that have 
been gamed since. Even though transaction costs theory and contract theory have made 
significant steps to explain the formation of inter-firm collaboration, it cannot be argued that 
formal mainstream economic theory has been able to approach the richness of concepts arising in 
the fields of strategic management and evolutionary economics. A combinatorial approach that 
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would reach across narrow disciplinary approaches is considered more appropriate for the 
methodological approach to SRP indicators. 

• A number of definitions, taxonomies, and analytical approaches to SRPs have been posited in the 
literature. Although academically interesting, such diversity also introduces difficulties in 
comparing analytical results across studies. 

• . The broadest possible differentiation of SRPs is between formal and informal relationships. They 

are both very valuable to supporting innovation. Informal relationships seem to be the larger but 
also the most amorphous of these two sets, presenting great difficulty in data collection. All 
existing systematic databases exclude informal SRPs. 

• Extant databases (CATI, CORE, NCRA-RJV) are as diverse as the definition of an SRP. Even 
though the basic intellectual interests that motivated their construction were fairly similar, there is 
a significant difference between the first and the other two due to the difference in sources of 
original material and data collection approach. 

• A major limitation of all existing databases on partnerships is the lack of information concerning 
the termination of partnerships. 

• Another major limitation of existing databases is the lack of performance measures for 
partnerships. This seems to be primarily the result of continuing uncertainty among experts 
concerning the level at which performance is judged (e.g., at the level of the partnership or the 
individual member) and the yardsticks that can be used. 

• Yet another major limitation may be that small firms may be underrepresented in most existing 
SRP databases. Possible reasons for that may be that small firms often do not formalize their 
R&D and probably their partnerships too. One implication is that an inference that SRPs are 
relatively less important for small firms than they are for large firms could well be mistaken. 
Another implication is that we may need to take a more serious look at informal partnerships than 
has been done in the past. There is thus some ground for arguing the necessity of a survey that 
would carefully reach small as well as larger firms. Some experts do, however, doubt the 
reliability of survey answers by a significant cohort of small firms, particularly those in non-R&D 
intensive activities. 

• Formal SRPs are just one of the ways firms may use to link to each other and to other 
organizations. Consequently, it is argued that SRP indicators will underestimate the extent of 
collaboration. To get a more complete picture, it may be useful to combine SRP indicators to 
others such as bibliometric and patent indicators. 

• CRADAs is an important form of SRPs with potentially a lot of information, which has yet to be 
collected extensively and systematically. SRS has been reporting aggregate data on CRADAs, 
but there is a sense that much broader coverage is warranted. Output indicators are also needed. 

• A second important differentiation of SRPs is between public-private and private-private. Which 
one of these is of interest crucially depends on the question. The presumption that public-private 
SRPs may be more important for public policy makes the implicit strong assumption that public 
policy is limited to contributing resources to SRPs. Policy decision-makers are clearly interested 
in other policies as well that affect the economic and regulatory environment in which SRPs 
operate. Knowledge about both sets of SRPs is useful for policy. 

• It would be useful to researchers and policy analysts if existing and to-be-created SRP indicators 
could be linked to other data related to economic and technological performance of individual 
organizations that other government agencies collect (e.g., market performance of individual 
firms and patent portfolios). Such bridging of data is expected to produce direct benefits to SRP 
analysis, not least because it will allow creating control groups of non-SRP participants. 
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• One often meets in the literature efforts to approximate the performance of SRPs with patents. A 
suggestion has been that collaboration in R&D should result in co-inventing and co-patenting. 
Unfortunately, co-patenting is much too little to be relied upon as such an indicator. It appears 
that firms have traditionally avoided co-patenting, probably due to expectations of increased legal 
problems. Although co-patenting numbers are increasing, they are still too low relative to the 
overall level of patents issued annually in the United States. 

• One other set of indicators that may be naturally linked to SRP indicators is the so-called 
innovation indicators. Such indicators are being constructed in the European Union and a few 
individual country members. NSF is also considering them and currently fielding a pilot survey 
in the information technology area. For example, one possibility could be to use the same survey 
instrument for both general innovation indicators and SRP indicators. There is some skepticism 
among experts concerning the feasibility and reliability of innovation indicators — having to do 
particularly with the definition of an innovation and the willingness of respondents to identify 
failed innovations. Still, in the lack of better alternatives, many consider this worth the try; it is 
expected that there is a learning curve in raising the quality of collected statistics. 

• NSF is clearly not the only agency considering SRP indicators. The Organization for Economic 
Cooperation and Development has also developed a strong interest in the subject, in relation to 
several other concurrent activities. The latter include: the revision of the Frascati manual to better 
define R&D, partnerships, outsourcing, etc.; the creation of a Globalization manual; the revision 
of the Oslo manual on innovation surveys. There is a strong sense at the OECD that this is the 
right time in the life cycle of the phenomenon for public agencies to get involved in defining and 
building statistical information on SRP indicators. 

• One of the more interesting international phenomena in SRPs during the past couple of decades 
has been the relative increase in the proportion of contractual agreements vis-a-vis traditional, 
equity-based joint ventures that create new organizational entities. This is an important 
development from both the business strategy and public policy points of view. It is also a major 
complicating factor in collecting data for SRP indicators. Many of these contractual agreements 
are synthetic. That is, these agreements include various activities, thus introducing a problem of 
classification, and apparently are of more informal nature than required for accurate reporting. 

• It is now clear that activity in three technological areas has been responsible for the vast majority 
of observed SRPs during the past couple of decades. The largest by far is information 
technology. Biotechnology comes next. Advanced materials follow. 

• SRP intensity and characteristics may vary with the field and the technological life cycle. A 
possible consideration is to link SRP data to some kind of a life cycle model. * . 

• Specifically to biotech, SRPs are extremely useful in the research, production, and delivery stages 
of products. Biotechnology is, of course, an area with particular characteristics, including greater 
than average reliance on academic research, proliferation of small firms, rapid turnover of firms, 
a strong effort by the big pharmaceuticals to avoid falling behind, extensive patentability of 
research outcomes, significant availability of venture capital. There are already four proprietary 
databases relating to biotechnology SRPs, with reported limitations in terms of scope of data, 
cost, and quality. 

• Biotechnology SRPs frequently emerge among people with prior working relationships. If the 
observation is accurate, as it seems to be, human (scientist) mobility and SRP formation 
indicators should correlate strongly. It would be interesting to investigate the relationship 
between human mobility and partnerships in other technology areas as well. 

• The proliferation of SRPs and the observation of a large failure rate of SRPs have naturally raised 
the question of performance. Early attempts to approximate SRP success by the longevity and 
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stability of the relationship were more appropriate for equity-based partnerships than for most 
cases of contractual agreements. It is also clear that different partners have different objectives in 
the same partnership, making the definition of partnership performance problematic. This has 
obvious implications for data and indicators of performance. It is quite possible that there are no 
widely applicable objective indicators of success at the level of the partnership. SRS may have to 
make do with indicators of success at the level of the individual partner. 

• A major complication in constructing partnership performance indicators reflects a selection 
problem. That is to say, if SRP participants tend to perform better than non-participants, is the 
difference the result of SRP participation or the result of SRP participants performing better 
irrespective of membership? 

• Two important features of SRPs that have received a lot of attention by business analysts and 
much less so by all others is the multi-dimensionality of relationships and their organizational 
structure. As far as NSF’s effort is concerned, the danger is with expanding the scope of the 
indicators very much to make them intractable and with crossing the borders between formal and 
informal SRPs, which increases the degree of fuzziness. 

• The construction of SRP indicators must take into account the strong current interest in 
innovation environments and in the increasing role of networks as the reference point for 
individual innovations. Significant effort must be devoted to argue over the appropriate unit of 
analysis which could be the individual, the group, the business unit, or the firm. Ideally, one 
would like to have information at different levels. 

IV. Developing SRP Indicators by SRS 

It emerged from the background papers and the expert discussion during the workshop that: 

• SRPs have important effects on innovative investment behavior and performance; 

• Public policy can affect the performance of SRPs, both directly in certain circumstances and 
indirectly by changing the rules of the game; 

• Currently available data on SRPs are limited in several respects, not systematically gathered or 
coordinated, and uneven in quality and degree of coverage. 

Experts believe that there is a need for ongoing, systematic, and coordinated documentation and 
reportingof: 

• Incidence of SRPs 

• Types of SRPs 

• Inputs into the formation of SRPs 

• Output of SRPs 

• Role of SRPs as inputs into the innovation process 

There is a clear sense that multiple measures of inputs and outputs, each reflecting the particular 
circumstances of a type of SRP, are useful and appropriate. Appropriate data could and should be 
integrated from a variety of sources in order to be useful in answering various, and quite different, 
policy- and strategy-oriented questions that analysts ask in relation to SRPs., 

It would certainly prove useful to have information on: 

• number, industrial affiliation, and size of participating firms in SRPs 

• • organization structure of SRPs and types of members 

• overall budget and individual contributions to SRPs, including government cost-share 
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• reasons for the termination of SRPs 

• SRPs outputs, such as new products, patents, publications, licensing agreements, launching of 
startups, new jobs 

• Impact of the SRP experience on the performance of individual participants 

Such lists are limited only by the imagination of the analyst, however, and are often beyond the 
means of responsible agencies to collect. 

Four important considerations for SRS were discussed: 

• Is this the right time for direct SRS involvement in building SRP indicators? 

• What would these indicators be? 

• What indicators are also applicable in the international context? 

• What are the policy issues we are trying to inform? 

The discussion covered a wide range - it was meant to be brainstorming, after all. This was the first 
time that such a gathering took place in the United States. Nonetheless, several important indications 
were signaled to SRS representatives. They included: 

• On the question of whether coverage should be limited to public/private SRPs, or to private- 
private SRPs, or be extended to both kinds, the outcome was balanced towards the latter. 

• Public agency support of public/private SRPs should be collected. 

• Questions could be added to existing NSF surveys to elicit information on the extent of private 
organizations’ resources devoted to SRP activity. 

• Additional survey questions should address the formal versus informal nature of SRPs. 

• While the utility of a broad question asking respondents to estimate the share of formal and 
informal partnerships was felt to be significant, the difficulties in definitions and estimation seem 
to make it difficult to operationalize. 

• The human mobility aspect of SRPs have not been addressed by any of the existing SRP 
databases. 

• Data on international collaboration are important in spite of complicated definitional problems. 

• An argument was made over the distinction between and value of understanding the extent of 
collaboration and understanding the factors driving it. Budget limitations may oblige SRS to 
follow a stepwise approach, however, limiting the coverage to the first leg as a start. 

• Proceeding in parallel with survey data collection and with case studies of SRPs may reveal 
unanticipated trouble spots regarding definition, inputs, outputs, budgets, and so forth. 

• Additional information should be collected on the experience of international agencies in 
developing SRP indicators and building appropriate survey instruments. 

Although the workshop produced a useful dialogue, it was by no means conclusive. It clearly 
underlined the importance of a continuing, proactive discussion between experts and the SRS 
regarding the most appropriate procedure for developing SRP indicators. 
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Strategic Research Partnerships: Evidence and Analysis 



Stephen Martin 
University of Amsterdam 



Results? Why, man, I have gotten a lot of results. I know several thousand things that 
won’t work. 

— Edison 



I. Introduction 

A strategic alliance (Chan et al., 1997, pp. 199-200) “enables a firm to focus resources on its core 
skills and competencies while acquiring other components or capabilities it lacks from the 
marketplace.” Such alliances extend beyond research, the focus of this paper, and take a variety of 
forms. 1 

It is sometimes said that theoretical research in economics is a largely self-contained activity, firmly 
insulated from the vagaries of evidence about the subjects it analyzes. 2 The extent to which this might 
be true in general is beyond the scope of this paper, but it does seem to be an accurate description of a 
good deal of the recent theoretical literature on R&D cooperation. 

What is even more discouraging, however, is that much of the policy literature on R&D cooperation 
seems disconnected from both mainstream theoretical 3 and empirical work on the phenomenon. 

Much of the theoretical work on R&D cooperation in the 1990s took off from d’Aspremont and 
Jacquemin, AJ, (1988). The impact of this linear demand, quadratic cost of own-cost reduction, 
deterministic R&D model may well have surprised even the authors, who referred to it as “an 
example.” In its basic form, it considers a duopoly market. Since there are only two firms, R&D 
cooperation, if it occurs at all, includes all firms in the industry. This is in sharp contrast to the kind of 
R&D cooperation described by, for example, Hagedoom and Schakenraad (1993), who document that 
major players in innovation-intensive industries simultaneously undertake a great many cooperative 
R&D projects, often with narrowly defined targets. 

In the AJ model, innovation is deterministic: firms select a certain cost reduction that they will pay 
for, and they obtain that cost reduction with certainty. Since innovation is deterministic, if a firm 
undertakes R&D at all, it performs only one R&D project. A deterministic formulation abstracts from 



1 My own preference is for the “operating entity JV” and “secretariat JV” classification of Ouchi (1989) and 
Vonortas (1994), which at least has the merit of being based on functional differences. 

2 Leontief s (1982) comments are well known: “Page after page of professional economic journals are filled 
with mathematical formulas leading the reader from sets of more or less plausible but entirely arbitrary 
assumptions to precisely stated but irrelevant theoretical conclusions.” 

3 In some cases, at least, deliberately so (Teece, 1996, p. 194). 
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